最近，Experts（简称为MOE）体系结构在提高大规模语言模型的模型能力方面取得了巨大的成功。但是，MOE需要比要扩展的基本模型要合并更多的参数。在本文中，我们建议通过跨专家共享信息来构建一个有效的MOE架构。我们采用矩阵产品运营商（MPO，量子多体物理学的张量分解）来重建专家层中的参数矩阵，并通过共享中央张量的参数（包含核心信息）来增加预训练语言模型的模型容量（ ）在不同专家的同时，通过不同专家的辅助张量（补充中央张量）实现特异性。为了解决不平衡的优化问题，我们进一步设计了基于MPO的MOE体系结构的梯度面膜策略。基于T5和GPT-2的广泛实验表明，预训练的语言模型的性能和效率提高（与开关变压器相比，高级模型性能的总参数降低了27.2倍）。我们的代码可在\ url {https://github.com/rucaibox/mpo/mpoe}上公开获得。

Event cameras that asynchronously output low-latency event streams provide great opportunities for state estimation under challenging situations. Despite event-based visual odometry having been extensively studied in recent years, most of them are based on monocular and few research on stereo event vision. In this paper, we present ESVIO, the first event-based stereo visual-inertial odometry, which leverages the complementary advantages of event streams, standard images and inertial measurements. Our proposed pipeline achieves temporal tracking and instantaneous matching between consecutive stereo event streams, thereby obtaining robust state estimation. In addition, the motion compensation method is designed to emphasize the edge of scenes by warping each event to reference moments with IMU and ESVIO back-end. We validate that both ESIO (purely event-based) and ESVIO (event with image-aided) have superior performance compared with other image-based and event-based baseline methods on public and self-collected datasets. Furthermore, we use our pipeline to perform onboard quadrotor flights under low-light environments. A real-world large-scale experiment is also conducted to demonstrate long-term effectiveness. We highlight that this work is a real-time, accurate system that is aimed at robust state estimation under challenging environments.

With the rising need of interpretable machine learning methods, there is a necessity for a rise in human effort to provide diverse explanations of the influencing factors of the model decisions. To improve the trust and transparency of AI-based systems, the EXplainable Artificial Intelligence (XAI) field has emerged. The XAI paradigm is bifurcated into two main categories: feature attribution and counterfactual explanation methods. While feature attribution methods are based on explaining the reason behind a model decision, counterfactual explanation methods discover the smallest input changes that will result in a different decision. In this paper, we aim at building trust and transparency in time series models by using motifs to generate counterfactual explanations. We propose Motif-Guided Counterfactual Explanation (MG-CF), a novel model that generates intuitive post-hoc counterfactual explanations that make full use of important motifs to provide interpretive information in decision-making processes. To the best of our knowledge, this is the first effort that leverages motifs to guide the counterfactual explanation generation. We validated our model using five real-world time-series datasets from the UCR repository. Our experimental results show the superiority of MG-CF in balancing all the desirable counterfactual explanations properties in comparison with other competing state-of-the-art baselines.

With the spread of tampered images, locating the tampered regions in digital images has drawn increasing attention. The existing image tampering localization methods, however, suffer from severe performance degradation when the tampered images are subjected to some post-processing, as the tampering traces would be distorted by the post-processing operations. The poor robustness against post-processing has become a bottleneck for the practical applications of image tampering localization techniques. In order to address this issue, this paper proposes a novel restoration-assisted framework for image tampering localization (ReLoc). The ReLoc framework mainly consists of an image restoration module and a tampering localization module. The key idea of ReLoc is to use the restoration module to recover a high-quality counterpart of the distorted tampered image, such that the distorted tampering traces can be re-enhanced, facilitating the tampering localization module to identify the tampered regions. To achieve this, the restoration module is optimized not only with the conventional constraints on image visual quality but also with a forensics-oriented objective function. Furthermore, the restoration module and the localization module are trained alternately, which can stabilize the training process and is beneficial for improving the performance. The proposed framework is evaluated by fighting against JPEG compression, the most commonly used post-processing. Extensive experimental results show that ReLoc can significantly improve the robustness against JPEG compression. The restoration module in a well-trained ReLoc model is transferable. Namely, it is still effective when being directly deployed with another tampering localization module.

事件摄像机是运动激活的传感器，可捕获像素级照明的变化，而不是具有固定帧速率的强度图像。与标准摄像机相比，它可以在高速运动和高动态范围场景中提供可靠的视觉感知。但是，当相机和场景之间的相对运动受到限制时，例如在静态状态下，事件摄像机仅输出一点信息甚至噪音。尽管标准相机可以在大多数情况下，尤其是在良好的照明条件下提供丰富的感知信息。这两个相机完全是互补的。在本文中，我们提出了一种具有鲁棒性，高智能和实时优化的基于事件的视觉惯性镜（VIO）方法，具有事件角度，基于线的事件功能和基于点的图像功能。提出的方法旨在利用人为场景中的自然场景和基于线路的功能中的基于点的功能，以通过设计良好设计的功能管理提供更多其他结构或约束信息。公共基准数据集中的实验表明，与基于图像或基于事件的VIO相比，我们的方法可以实现卓越的性能。最后，我们使用我们的方法演示了机上闭环自动驾驶四极管飞行和大规模室外实验。评估的视频在我们的项目网站上介绍：https：//b23.tv/oe3qm6j

潜在空间基于能量的模型（EBM），也称为基于能量的先验，引起了对生成建模的日益兴趣。由于其在潜在空间的配方和强大的建模能力方面的灵活性所推动，最近构建的作品已经进行了有趣的尝试，目的是针对文本建模的解释性。但是，潜在空间EBM还继承了数据空间中EBM的一些缺陷。实践中退化的MCMC抽样质量会导致培训中的发电质量和不稳定差，尤其是在具有复杂潜在结构的数据上。受到最近的努力的启发，该努力利用扩散恢复的可能性学习是解决抽样问题的一种方法，我们在变异学习框架中引入了扩散模型和潜在空间EBM之间的新型共生，这是潜在扩散能量基于能量的模型。我们与信息瓶颈共同开发基于几何聚类的正则化，以进一步提高学到的潜在空间的质量。对几个具有挑战性的任务进行的实验证明了我们模型在可解释的文本建模上的优越性能而不是强大的同行。

我们呈现深度区域竞争（DRC），这是一种旨在以完全无监督的方式从图像中提取前景对象的算法。前景提取可以被视为一种特殊的泛型图像分段的情况，专注于从背景中识别和解开对象。在这项工作中，我们通过以专家（MOE）的混合形式的生成图像建模和生成图像建模来重新思考前景提取，我们进一步介绍了学习的像素重新分配作为捕获规律的基本诱导偏差背景区域。通过这种建模，可以通过期望最大化（EM）自然地发现前景背景分区。我们表明，该方法有效利用了在分区过程中混合成分之间的相互作用，该分区过程紧密地连接到区域竞争，是通用图像分割的一个精细方法。实验表明，与现有方法相比，DRC在复杂的真实数据上表现出更具竞争力的性能和具有挑战性的多对象场景。此外，我们认为，即使在训练期间看不见的类别，DRC也可能概括为新的前景物体。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.